SIMONCELLI Eero NYU METAMERS Jan 17 2012

I was abt 8 mins late;

most of sense info we discard;
some we act on immed;  
some we remember

Metamers:  2 stim that appear the same
(Eero first learned of this from Brian Wandell ... in our audience)

Newton 1665... spectrum... splitting light into colors...

Young, Helmholtz, Grassman etc  from 1800s

take an arbitrary test light  eg  green... now match that
with 3 knobs  to control intensity of each of red, green, blue...

and now the lights look identical  (even tho they have difft mixtures of light)

so how to make sense of this...

so,  brain is  (perhaps, one theorizes)  linear combo of  the 3 lights
and all we hold onto is the combo

that accurately predicts perceptual limitations  and is
the basis for color tech standards  (as in CIE 1931)

Dennis Baylor (my old neighbor)  1981  
sucked indiv cones into pipette and did indiv  wave lengths...

so, there is a loss of info  (with color)  now

switch to SPATIAL INFO  (where does the info go)

info comes in and splits in ventral and dorsal pathway (v1 v3 v5  for position, motion, action
ventral is v1 v2 v4 IT;

Fact 1:  Receptive Fields  RF  (RGCs of V1 cells)

Fact 2  they tile entire field

Fact 3:  RF  sizes grow with eccentricity

RGC  (midget)  cell receptive fields
he shows macaque retina, magnified by 10)

true in the retina and in v1, v2, v4 as well

in Macaque:  see his just grad PhD student  Jeremy Freeman and Simoncelli 2011

increasing slope of size of RFs with eccentricity  (by the time u get to IT the RFs are huge)

So,  what is being measured?

goes back to Hubel and Wiesel 1962  (and before to Steve. Kuffler)

like a V1 simple cell  he shows

he makes a simple cell that responds to a bar  by lining up  3 RGC cells synapsing to one simple cell;

Conjecture:  canonical cortical computation

simple to complex cells at each stage, all the way up

this is a dimension reduction... (info is being thrown away)

what is being thrown away...  texture!

Julesz (1962)

hypothesis:  2 textures with identical Nth order pixel statistics will  look the same (for some N)...
same idea as metamer  (statistics are a summary of input... so any data that produce the same stats shud look the same...
but  Julesz  discarded it cuz he felt he disproved it with a counterexample...

he shows examples from Portilla and Simoncelli 2000
he shows textures of  clothes, pebbles, sand, pebbles, richard feynman....

he shows a great demo...  he shows a large b and w photo of a crowd looking at a performer in Wash Square Park...
and we all stare at the center fixation point... he then flips back and forth btwn orig image and a distorted image and

we cannot see the very distorted periphery  (the periphery only retains say first N dimentions of statistics)

can tweak the Model RF scaling (diameter vs eccentricity)

when scaling is really low then it looks identical...

V2  has a RF abt twice v1  and V4  is abt twice V4...

he spends a lot of time on Freeman and Simoncelli  Nat Neurosci 2011...

grand delusion... you think you're taking it all in... but
in fact, there is no pictorial repr of your visual world... you reassure yourself  by shifting gaze....

perhaps can design written material to compensate for this loss of eccentric info...

he shows text that has also been distorted eccentrically.

********

Camouflage.... soldier gets lost in background....

Tony Movshon is down the hall from Eero at NYU.

how can we use this to figure out how to drive the cells in V2... Answer is texture...

he shows 2 rows of images with textures
V2 cares abt higher order statistics of texture...

he shows macaque firing data in V2...
v big incr in response in V2 cells....

we've got a stimulus set that will drive V2 cells..

with david heeger... compared macaque to human (fMRI response in V2)
(btwn  texture vs  spectral noise with equiv  power)

summary:

Ventral Stream Metamers

provides a model driven explanation for crowding...

similar to Ruth Rosenholtz  2012...

*****
also tried this with Auditory system!

needed a model for audit texture...

they make a mid-level auditory  model...

he plays auditory texture examples

bubbling water:   original and synthesized

insects orig vs synthetic... similar sound

and rustling paper... they sound identical

synthezie a new sound that hs the same statistics... and they sound the same...

McDermott and Simoncelli 2011

full model has marginals, correla;tions, modulation power... etc...

just accepted paper:  McDermott and Simoncelli to appear in Nat Neurosci...

how long does it take to do this averaging>  this is with sounds....

l half of slide:  the more info the better (listening fo 2.5 sec is better than 40 ms...

BUT in R half of slide, he shows an expt in which  the longer u listen, the worse u get,
cuz.......

*****

CAST:  Javier Portilla,  Jeremy Freeman

V2 physio  Corey Ziemba, Jermey Freeman, Tony Movshon

Q (from Bill Newsome):  u say u don't have a model... what do u mean?
A  he means a model as to how the system works... how it learns to use those statistics... how it is wired to learn this...

what does texture have to do when I'm looking at an object?